Electronic system with eye protection in response to user distance

ABSTRACT

An electronic system including a display device, an image sensor, a face detection engine, an eye detection engine and an eye protection engine is provided. The image sensor captures an image. The face detection engine recognizes a user face in the image. The eye detection engine recognizes user eyes in the image. The eye protection engine turns off the display device when the user eyes are recognized in the image but the user face is not recognized in the image.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation application of U.S. patentapplication Ser. No. 17/515,739 filed on, Nov. 1, 2021, which is acontinuation application of U.S. patent application Ser. No. 17/021,372filed on, Sep. 15, 2020, which is a continuation application of U.S.patent application Ser. No. 16/717,174 filed on, Dec. 17, 2019, which isa continuation application of U.S. patent application Ser. No.15/821,891 filed on, Nov. 24, 2017, which is a continuation applicationof U.S. patent application Ser. No. 15/253,453 filed on, Aug. 31, 2016,which is continuation-in-part application of U.S. patent applicationSer. No. 13/934,311 filed on, Jul. 3, 2013, and claims the prioritybenefit of Taiwan Patent Application Serial Number 101126421, filed onJul. 20, 2012, the full disclosures of which are incorporated herein byreference.

To the extent any amendments, characterizations, or other assertionspreviously made (in this or in any related patent applications orpatents, including any parent, sibling, or child) with respect to anyart, prior or otherwise, could be construed as a disclaimer of anysubject matter supported by the present disclosure of this application,Applicant hereby rescinds and retracts such disclaimer. Applicant alsorespectfully submits that any prior art previously considered in anyrelated patent applications or patents, including any parent, sibling,or child, may need to be re-visited.

BACKGROUND 1. Field of the Disclosure

This disclosure generally relates to an image system and, moreparticularly, to an image system with the eye protection and anoperating method thereof.

2. Description of the Related Art

Interactive control mechanism can provide users a more instinctivecontrol and thus it has been broadly applied to various multimediasystems, especially to an image display system having a display screen.

It is a general method to use a remote controller capable of capturingimages as an interactive human machine interface, and the remotecontroller can be manufactured as various properties, such as a bat, aracket and a club. Another kind of human machine interface may beoperated without using any handheld device. For example, a pupiltracking device may perform the interactive operation according to theline of sight change of a user.

Referring to FIGS. 1A and 1B, FIG. 1A shows a conventional pupiltracking system which is configured to perform the pupil tracking of ahuman eye 9; and FIG. 1B shows a schematic diagram of the image of humaneye captured by the conventional pupil tracking system. The pupiltracking system includes a display device 81, a light source 82, animage sensor 83 and a processing unit 84. The light source 82 isconfigured to emit light toward the human eye 9 so as to form a lightimage I₈₂ in the human eye 9 as shown in FIG. 1B. The image sensor 83 isconfigured to capture an image of human eye containing a pupil 91 andthe light image 182, and the processing unit 84 is configured tocalculate the variation of a relative distance D between the pupil 91and the light image I₈₂ in the image of human eye so as to track thepupil 91 and to accordingly control the motion of a cursor 811 shown onthe display device 81. However, if there is another ambient light (notshown) forming an ambient light image I₀ in the image of human eye,errors can occur in pupil tracking.

In addition, the time that people operate a device having a displayscreen within a day becomes longer and longer, and thus the protectionof human eyes becomes more important.

SUMMARY

The present disclosure provides an image system with the eye protectionand an operating method thereof that automatically switch off a displayscreen according to an operating distance of a user and/or ambient lightstrength.

The present disclosure provides an electronic system including an imagesensor, a display screen and a processor. The image sensor has a fieldof view and is configured to capture an image. The processor iselectrically coupled to the image sensor and the display screen, andconfigured to perform a face detection and an eye detection on theimage, perform a notice when the image is recognized to contain eyes butdoes not contain a complete face, and switch off the display screen whenthe notice is not turned off within a predetermined time interval.

The present disclosure further provides an electronic system includingan image sensor, a processor and a display screen. The image sensor hasa field of view and is configured to capture an image. The processor iselectrically coupled to the image sensor, and configured to perform aface detection and an eye detection on the image. The display screen isconfigured to display with a brightness when the image is recognized tocontain both eyes and a complete face, and reduce the brightness whenthe image is recognized to contain the eyes but does not contain thecomplete face.

The present disclosure further provides an electronic system includingan image sensor, a display screen and a processor. The image sensor hasa field of view and is configured to capture an image. The processor iselectrically coupled to the image sensor and the display screen, andconfigured to perform a face detection and an eye detection on theimage, and control a speaker to play a warning voice when the image isrecognized to contain eyes but does not contain a complete face.

BRIEF DESCRIPTION OF THE DRAWINGS

Other objects, advantages, and novel features of the present disclosurewill become more apparent from the following detailed description whentaken in conjunction with the accompanying drawings.

FIG. 1A shows a schematic diagram of the conventional pupil trackingsystem.

FIG. 1B shows a schematic diagram of the image of human eye captured bythe conventional pupil tracking system.

FIG. 2 shows an operational schematic diagram of the pupil detectiondevice according to an embodiment of the present disclosure.

FIGS. 3A-3C show schematic diagrams of the image capturing and thelighting of the light source in the pupil detection device according tothe embodiment of the present disclosure.

FIG. 4 shows a schematic diagram of performing the pupil detectionaccording to an image to be identified captured by the pupil detectiondevice according to the embodiment of the present disclosure.

FIG. 5A shows an operational schematic diagram of the pupil detectiondevice according to another embodiment of the present disclosure.

FIG. 5B shows an operational schematic diagram of the pupil detectiondevice according to an alternative embodiment of the present disclosure.

FIG. 6 is a schematic block diagram of an image system according to oneembodiment of the present disclosure.

FIG. 7 is a schematic diagram of an image system according to oneembodiment of the present disclosure.

FIGS. 8A and 8B are images captured by an image sensor of an imagesystem according to some embodiments of the present disclosure.

FIG. 9 is a flow chart of an operating method of an image systemaccording to one embodiment of the present disclosure.

FIG. 10 is a flow chart of an operating method of an image systemaccording to an alternative embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENT

It should be noted that, wherever possible, the same reference numberswill be used throughout the drawings to refer to the same or like parts.

Referring to FIG. 2 , it shows an operational schematic diagram of thepupil detection device 1 according to an embodiment of the presentdisclosure. The pupil detection device 1 is configured to detect a pupilposition of an eyeball 90 and to output a pupil coordinate associatedwith the pupil position. The pupil detection device 1 includes an activelight source 11, an image sensor 12 and a processing unit 13. Generallyspeaking, when the eyeball 90 looks downward the eyelid may cover a partof the eyeball 90. Therefore, if the pupil detection device 1 isdisposed on a head accessory 2, a disposed position of the image sensor12 is preferably lower than the eyeball 90. For example in FIG. 2 , whenthe pupil detection device 1 is disposed on eyeglasses or a goggle, thepupil detection device 1 is preferably disposed at the lower framethereof such that the pupil can be detected even though the eyeball 90looks downward (i.e. the pupil directing downward).

The active light source 11 may be an infrared light source, e.g. aninfrared light emitting diode, in order not to influence the line ofsight when lighting. The active light source 11 emits light toward theeyeball 90. It should be mentioned that the active light source 11 maybe a single light source or formed by arranging a plurality of lightsources.

The image sensor 12 may be a photosensor configured to sense opticalenergy, such as a CCD image sensor, a CMOS image sensor or the like. Theimage sensor 12 captures at least one image frame of the eyeball 90 witha resolution and the captured image frame is served as an image to beidentified.

For example referring to FIGS. 3A-3C and 4 , FIGS. 3A-3C show schematicdiagrams of the image capturing of the image sensor 12 and the lightingof the active light source 11; and FIG. 4 shows a schematic diagram ofperforming the pupil detection according to the image to be identifiedcaptured by the image sensor 12. The image sensor 12 captures imageframes of the eyeball 90 at a frame rate to be served as images to beidentified F. In one embodiment, the active light source 11 emits lightwith a fixed brightness value and corresponding to the image capturingof the image sensor 12 (FIG. 3B), and the image sensor 12 sequentiallyoutputs image frames f to be served as the image to be identified F(i.e. F=f), wherein the image to be identified F may include a pupil 91,an iris 92 and the white of the eye 93. In another embodiment, theactive light source 11 emits light alternatively in a first brightnessvalue and a second brightness value, and the image sensor 12 captures afirst image frame f1 corresponding to the first brightness value and asecond image frame f2 corresponding to the second brightness value (FIG.3C). The processing unit 13 calculates a differential image (f1−f2) ofthe first image frame f1 and the second image frame f2 to be served asthe image to be identified F; i.e. F=(f1−f2). It should be mentionedthat the first brightness value may not be equal to the secondbrightness value and both brightness values are not equal to zero.Accordingly, the processing unit 13 may eliminate the influence fromambient light sources by calculating the differential image (f1−f2).

The processing unit 13 may be a digital signal processor (DSP), and isconfigured to calculate a minimum gray value P₁ in the image to beidentified F and to identify a plurality of pixels surrounding theminimum gray value P₁ and having gray values within a gray value rangeRg as a pupil area PA, as shown in FIG. 4 . When the active light source11 turns on, as the pupil 91 has a lowest brightness value, the white ofthe eye 93 has a highest brightness value and the iris 92 has abrightness value between that of the pupil 91 and the white of the eye93, the minimum gray value P₁ will appear inside the pupil 91.Therefore, a pixel area surrounding the minimum gray value P₁ may beidentified as the pupil area PA, and the pixel area neighboring theminimum gray value P₁ may be correlated as a single object using theimage grouping technique, which may be referred to U.S. PatentPublication No. 2011/0176733, entitled “image recognition method” andassigned to the same assignee of the present application, and the fulldisclosure of which is incorporated herein by reference. In addition,the setting of the gray value range Rg may be adjusted according to theoperation environment of the pupil detection device 1, e.g. differentgray value ranges Rg may be set for indoors and outdoors. Furthermore,in order to eliminate the noise interference, the processing unit 13 mayfurther identify whether the pupil area PA is an image of ambient lightsource according to its features such as the size and shape thereof. Forexample, if the pupil area PA is too small or not an approximate circle,it may be an image of ambient light source and can be removed.

Next, the processing unit 13 may calculate a gravity center or a centerof the pupil area PA to be served as a pupil position P₂ and output apupil coordinate (x,y) associated with the pupil position P₂. Theprocessing unit 13 may relatively control the motion of a cursor 811shown on a display device 81 according to the pupil coordinate (x,y). Itis appreciated that the pupil position P₂ may not be the same as aposition of the minimum gray value P₁.

In addition, as the pupil detection device 1 may be configured tocontrol an electronic device, in some cases the pupil detection device 1may preferably recognize the user ID so as to increase thepracticability or realize the privacy protection. Therefore, theprocessing unit 13 may perform the iris recognition according to theimage to be identified F. In this case the pupil detection device 1 mayfurther include a memory unit 14 configured to save the iris informationof different users. In addition, as the iris recognition needs a higherimage resolution and the pupil area identification needs a lower imageresolution, in this embodiment a resolution and a frame rate of theimage sensor 12 may be adjustable. For example, when the processing unit13 is configured to perform the iris recognition (e.g. a second mode),the image sensor 12 may capture image frames with a first resolution anda first frame rate, whereas when the processing unit 13 is configured toidentify the pupil area (e.g. a first mode), the image sensor 12 maycapture image frames with a second resolution and a second frame rate,wherein the first resolution may be higher than the second resolutionand the first frame rate may be lower than the second frame rate. Inthis embodiment, an adjustable range of the image resolution may bebetween 640×480 and 160×120, and an adjustable range of the frame ratemay be between 30 FPS and 480 FPS (frame/second), but the presentdisclosure is not limited thereto.

In this embodiment, as the processing unit 13 performs the pupildetection based on the minimum gray value in the eyeball image, it isable to eliminate the interference from ambient light sources since theambient light image has a higher gray value. In addition, it is able tofurther eliminate the ambient light image by calculating thedifferential image.

In another embodiment, the pupil detection device 1 may include morethan two image sensors configured to capture image frames of the sameeyeball and to accordingly calculate a three-dimensional pupil positionand cover a larger detection range; i.e. the two image sensorsconfigured to capture image frames of the same eyeball may be separatedby a predetermined distance.

Referring to FIG. 5A, it shows a schematic diagram of the pupildetection device 1 according to another embodiment of the presentdisclosure. Although the pupil detection device 1 is shown to bedisposed on eyeglasses, the present disclosure is not limited thereto.The pupil detection device 1 includes at least one active light source11, two image sensors 12, 12′ and a processing unit 13. It should bementioned that in this embodiment a plurality of active light sources 11may be used to improve the illumination (e.g. the active light source 11may be formed by arranging a plurality of light sources); and a numberof the image sensors 12, 12′ is not limited to two. If three, four ormore image sensors are included, each of the image sensors operatessimilar to the image sensors 12, 12′ and only their disposed positionsare different. However, their disposed positions are also preferablylower than the human eye 9. In addition, although the pupil detectiondevice 1 is shown to be arranged corresponding to the left eye 9L, itmay also be arranged corresponding to the right eye 9R. That is, if thepupil detection device 1 is disposed on a head accessory 2, the twoimage sensors 12, 12′ are preferably disposed lower than the left eye 9Lor the right eye 9R.

The at least one active light source 11 emits light to illuminate a lefteye 9L or a right eye 9R. The two image sensors 12, 12′ capture, with aresolution, at least one image frame of the left eye 9L or the right eye9R which is illuminated by the at least one active light source 11 to beserved as a first image to be identified F and a second image to beidentified F′, wherein the two image sensors 12, 12′ may or may notcapture the image frames simultaneously. The processing unit 13 isconfigured to respectively calculate a minimum gray value P₁ in thefirst image to be identified F and the second image to be identified F′,and to identify a plurality of pixels surrounding the minimum pixelvalue P₁ and having gray values within a gray value range Rg as a pupilarea PA. After the pupil area PA is obtained, the processing unit 13 isfurther configured to calculate a gravity center or a center of thepupil area PA to be served as a pupil position P₂ as shown in FIG. 4 andto output a left pupil coordinate L(x,y) and a right pupil coordinateR(x,y). In this embodiment, as the pupil is detected by using two imagesto be identified F, F′, the processing unit 13 may calculate athree-dimensional pupil position according to the pupil position P₂ inthe first image to be identified F and the second image to be identifiedF′. For example, the two image sensors 12, 12′ may be respectivelydisposed at two sides of a center line of the human eye 9, and theprocessing unit 13 may calculate the three-dimensional pupil positionaccording to the two images to be identified F, F′.

As mentioned above, in order to eliminate the ambient light image, theprocessing unit 13 may respectively calculate a differential image atfirst and then identify the pupil area PA according to the differentialimage. In this case the at least one active light source 11 emits lightalternatively in a first brightness value and a second brightness value;the two image sensors 12, 12′ capture a first image frame f1corresponding to the first brightness value and a second image frame f2corresponding to the second brightness value (as shown in FIG. 3C); andthe processing unit 13 may calculate a differential image (f1−f2) of thefirst image frame f1 and the second image frame f2 to be served as thefirst image to be identified F and the second image to be identified F′.

Similarly, in this embodiment the processing unit 13 may perform theiris recognition according to the first image to be identified F and/orthe second image to be identified F′. When the processing unit 13 isconfigured to perform the iris recognition, the image sensor 12 capturesimage frames with a first resolution and a first frame rate, whereaswhen the processing unit 13 is configured to identify the pupil area,the image sensor 12 captures image frames with a second resolution and asecond frame rate, wherein the first resolution may be higher than thesecond resolution, whereas the first frame rate may be lower than thesecond frame rate.

In another embodiment, the pupil detection device 1 may include morethan two image sensors configured to respectively capture image framesof different eyes so as to output the detection result of the left eyeand/or the right eye according to different conditions.

Referring to FIG. 5B, it shows a schematic diagram of the pupildetection device 1 according to an alternative embodiment of the presentdisclosure. Although the pupil detection device 1 is shown to bedisposed on a goggle, but the present disclosure is not limited thereto.The pupil detection device 1 includes two active light sources 11, 11′,two image sensors 12, 12′ and a processing unit 13. It should bementioned that more than one active light source may be usedcorresponding to each human eye so as to improve the illumination; and aplurality of image sensors may be used corresponding to each human eye(as shown in FIG. 5A). Similarly, if the pupil detection device 1 isdisposed on a head accessory 2, disposed positions of the two imagesensors 12, 12′ are preferably lower than the left eye 9L and the righteye 9R.

The two active light sources 11, 11′ emit light to respectivelyilluminate a left eye 9L and a right eye 9R. The two image sensors 12,12′ respectively capture, with a resolution, at least one image frame ofthe left eye 9L and the right eye 9R to be served as a first image to beidentified F and a second image to be identified F′. The processing unit13 is configured to respectively calculate a minimum gray value P₁ inthe first image to be identified F and the second image to be identifiedF′, and to identify a plurality of pixels surrounding the minimum grayvalue P₁ and having gray values within a gray value range Rg as a pupilarea PA. After the pupil area PA is obtained, the processing unit 13 maycalculate a gravity center or a center of the pupil area PA to be servedas a pupil position P₂ (as shown in FIG. 4 ) and output a left pupilcoordinate L(x,y) and a right pupil coordinate R(x,y). As two pupils arerespectively detected using different images to be identified in thisembodiment, coordinates of the two pupils may be respectively calculatedand different pupil coordinates may be outputted according to differentconditions. For example when the human eye looks rightward, the left eye9L may be blocked by the nose bridge and not be able to see the objectat the right hand side, the processing unit 13 may only calculate aright pupil coordinate R(x,y) associated with the right eye 9R accordingto the pupil position. For example when the human eye looks leftward,the right eye 9R may be blocked by the nose bridge and not be able tosee the object at the left hand side, the processing unit 13 may onlycalculate a left pupil coordinate L(x,y) associated with the left eye 9Laccording to the pupil position. In other conditions the processing unit13 may calculate an average pupil coordinate associated with the lefteye 9L and the right eye 9R according to the pupil position. The presentdisclosure is not limited to the conditions above.

In another embodiment, it is able to estimate a gaze direction or a gazedistance according to the relationship between the left pupil coordinateL(x,y) and the right pupil coordinate R(x,y).

In another embodiment, if more than two image sensors are respectivelyarranged corresponding to the left eye 9L and the right eye 9R,three-dimensional pupil positions of the left eye 9L and the right eye9R may be respectively obtained.

As mentioned above, in order to eliminate the ambient light image, theprocessing unit 13 may respectively calculate a differential image atfirst and then identify the pupil area PA according to the differentialimage. In this case the two active light sources 11 emit lightalternatively in a first brightness value and a second brightness value;the two image sensors 12, 12′ capture a first image frame f1corresponding to the first brightness value and a second image frame f2corresponding to the second brightness value (as shown in FIG. 3C); andthe processing unit 13 may calculate a differential image (f1−f2) of thefirst image frame f1 and the second image frame f2 to be served as thefirst image to be identified F and the second image to be identified F′.

Similarly, in this embodiment the processing unit 13 may perform theiris recognition according to the first image to be identified F and/orthe second image to be identified F′. When the processing unit 13 isconfigured to perform the iris recognition, the image sensors 12, 12′may capture image frames with a first resolution and a first frame rate,whereas when the processing unit 13 is configured to identify the pupilarea, the image sensors 12, 12′ may capture image frames with a secondresolution and a second frame rate, wherein the first resolution may behigher than the second resolution, whereas the first frame rate may belower than the second frame rate.

In addition, the pupil detection device 1 of each embodiment of thepresent disclosure may cooperate with a display unit for displayingimages, and the display unit may also be disposed on the head accessory2, such as eyeglasses or a goggle.

The pupil detection device 1 of each embodiment of the presentdisclosure may further have the function of blinking detection. Forexample, the processing unit 13 may record time intervals during whichthe pupil is detected and is not detected so as to identify the blinkingoperation.

The pupil detection device 1 of each embodiment of the presentdisclosure may further have the function of doze detection anddistraction detection. For example, when the pupil detection device 1 isapplied to a vehicle device, it is able to detect whether the driver issleepy or pays attention to a forward direction and to give a warning ata proper time. The doze detection may be implemented by detecting a timeratio between eye open and eye close. The distraction detection may beimplemented by detecting a gaze direction of the driver.

The pupil detection device 1 of each embodiment of the presentdisclosure may further have the function of blinking frequency detectionand dry eye detection. Specifically speaking, the processing unit 13 mayestimate the possibility and degree of the dry eye according to thedetected blinking frequency and then remind the user to blink his or hereyes.

The pupil detection device 1 of each embodiment of the presentdisclosure may further have the function of gesture recognition. Thegesture recognition may be performed by moving the pupil toward apredetermined direction for a predetermined times and comparing thepupil movement with a predetermined gesture so as to execute specificfunctions. The gesture recognition is similar to those performed byother objects rather than the pupil, such as the gesture recognitionperformed by a hand motion or a finger motion.

The pupil detection device 1 of each embodiment of the presentdisclosure may further have the function of power saving. For example,the power save mode may be entered if the pupil is not detected for apredetermined time interval or the image variation of the image to beidentified is too small.

It should be mentioned that the pupil detection device 1 of eachembodiment of the present disclosure may be directly manufactured as ahead pupil detection device or be attached to a head accessory, e.g.eyeglasses, a goggle or a hat edge via a combining element. In otherembodiments, the pupil detection device 1 of each embodiment of thepresent disclosure may be disposed at other positions for performing thepupil detection, e.g. disposed in a car and close to the user's eyes(e.g. on a rearview mirror) as long as it is disposed at a positioncapable of detecting the human eye 9.

Referring to FIG. 6 , it is a schematic block diagram of an image systemaccording to one embodiment of the present disclosure. The image system600 includes an image sensor 61, a display screen 65 and a displaycontroller 63, wherein the display controller 63 is used to analyze animage Im captured by the image sensor 61 and control the display screen65, e.g., controlling the ON/OFF and/or displaying brightness. Thedisplay controller 63 is implemented by software, hardware, firmware ora combination thereof according to different applications. For example,the display controller 63 is a digital signal processor (DSP),microcontroller (MCU), an application specific integrated circuit(ASIC), a central processing unit (CPU) or the like.

The image system 600 is integrated on an electronic device having adisplay screen 65 as shown in FIG. 7 . Although the electronic device isshown as a cell phone in FIG. 7 , it is only intended to illustrate butnot to limit the present disclosure. The electronic device is, forexample, a cell phone, a smart phone, a personal digital assistance(PDA), a notebook computer, a tablet computer, a desktop computer, atelevision, a monitor, a vehicle-mounted central control system or thelike without particular limitations. The image system 600 has the eyeprotection function that closes the display screen 65 or reducesbrightness of the display screen 65 according to an operating distanceof a user and/or ambient light intensity. The display screen 65 operatesnormally when the user is at an appropriate distance from the displayscreen 65 and when ambient light is strong enough.

The image sensor 61 has a predetermined field of view (FOV) andconfigured to capture an image Im, e.g., acquiring an image containing apart of a human face in operation, at a sampling rate. The predeterminedfield of view (FOV) of the image sensor 61 is previously arrangedaccording to an operable range Do of the image system 600. For example,the operable range Do is arranged to allow a complete human face toappear within the FOV of the image sensor 61 when the human face isfarther than the operable range Do. For example, the operable range Dois preferably between 25 cm to 50 cm which may be determined accordingto a size of the display screen 65. For example, when the size of thedisplay screen 65 is large, a user may watch pictures shown on thedisplay screen 65 at a farther distance and thus the operable range Dois set longer; on the contrary, when the size of the display screen 65is small, the user watches pictures shown on the display screen 65 at ashorter distance and thus the operable range Do is set shorter. As longas the operable range Do is determined, it is able to design the FOV ofthe image sensor 61.

In the present disclosure, the image sensor 61 preferably has a lowresolution (e.g., hundreds of pixels), low sampling rate (e.g., fewhertz) and low power consumption. For example, the image sensor 61 is aCMOS image sensor having a sensor array having hundreds of pixels.Accordingly, the image sensor 61 may operate full-timely withoutconsuming too much power when the power of the image system 600 isswitched on. In some embodiments, the image sensor 61 is always turnedon and captures images at the sampling rate when a predeterminedApplication (APP) is executed by the image system 600. Saidpredetermined Application is, for example, a game, a video player, a webbrowser, an Office file or the like. Said predetermined Application isactivated, for example, by a user single clocking or double clicking anicon shown on the display screen 65 or by a voice instruction.

Since the image sensor 61 of the present disclosure is used to capturethe image Im exclusive to controlling the display screen 65, the imagesensor 61 is further set in addition to a conventional proximity sensorand a camera. For example referring to FIG. 7 , an image system 600(taking a smart phone as an example for illustration herein) includes aproximity sensor 67 for detecting whether an object is close to theimage system 600 and a camera 69 for capturing a picture to be displayedon the display screen 65 or stored in a memory. The proximity sensor 67senses information of light intensity, and the camera 69 has anextremely high resolution, e.g., millions of pixels. In the presentdisclosure, the image sensor 61 is not the proximity sensor 67 or thecamera 69.

The display screen 65 is a liquid crystal display (LCD), a plasmadisplay, an organic light emitting diode (OLED) display or the like fordisplaying pictures thereon. It is appreciated that those picturesdisplayed on the display screen 65 are determined according to the APP(software application) being run on the image system 600. The displayscreen 65 is controlled by the display controller 63 to be switched off.It should be mentioned that the “switch off” mentioned herein isreferred to turning off the backlight of the display screen 65, rotatingliquid crystal molecules to block the light penetration or cutting offthe electricity being provided to the display screen 65 or the likewithout particular limitations as long as there is no picture is shownthereon.

The display controller 63 is electrically coupled to the image sensor 61and the display screen 65, and used to control the display screen 65according to the image Im captured by the image sensor 61. The displaycontroller 63 is able to control the display screen 65 according to asingle image Im. The display controller 63 includes a face detectionengine 631, an eye detection engine 633 and an eye protection engine635. The face detection engine 631 and the eye detection engine 633 aremachine learning engines which are trained to recognize a face and eyesrespectively. For example, the face detection engine 631 and the eyedetection engine 633 are implemented by an adaptive boosting or aconvolution neural network, and trained before shipment of the imagesystem 600 to respectively recognize the face and eyes. As mentionedabove, the face detection engine 631 and the eye detection engine 633are implemented by software, hardware, firmware or a combinationthereof.

The face detection engine 631 is electrically coupled to the imagesensor 61 to receive an image Im therefrom. The face detection engine631 is used to recognize a face in the image Im. For example, the facedetection engine 631 sends a digital value “1” as a first control signalS_(c1) to the eye protection engine 635 when the face is recognized ordetected, and sends a digital value “0” as the first control signalS_(c1) to the eye protection engine 635 when the face is not recognizedor detected, but the present disclosure is not limited thereto. The facedetection engine 631 may transmit other digital values (e.g., two ormore bits) to inform the eye protection engine 635.

In some embodiments, a face being recognized herein is referred to acomplete face appearing within an image Im or within a window ofinterest (WOI) selected from the image Im. More specifically, the facedetection engine 631 is trained to identify a complete face in the imageIm or within the WOI selected from the image Im. In other embodiments, aface being recognized herein is referred to both the eyes and a mouthare recognized or detected within an image Im or within the WOI selectedfrom the image Im. The setting of a face being recognized (determinedaccording to the machine learning) directly affects the size of FOV andWOI and further determines the operable range Do of the image system600. Accordingly, it is able to previously determine the FOV or WOIaccording to an operable range Do before shipment, or to determine theoperable range Do by selecting an appropriate FOV or WOI beforeshipment.

The eye detection engine 633 is electrically coupled to the image sensor61 to receive an image Im therefrom. The eye detection engine 633 isused to recognize eyes in the image Im. For example, the eye detectionengine 633 sends a digital value “1” as a second control signal S_(c2)to the eye protection engine 635 when the eyes are recognized ordetected, and sends a digital value “0” as the second control signalS_(c2) to the eye protection engine 635 when the eyes are not recognizedor detected, but the present disclosure is not limited thereto. The eyedetection engine 633 may transmit other digital values (e.g., two ormore bits) to inform the eye protection engine 635. Similar to the facedetection engine 631, the eye detection engine 633 detects whether theeyes are within the image Im or within a window of interest of the imageIm.

The eye protection engine 635 is electrically coupled to the displayscreen 65, the face detection engine 631 and the eye detection engine633, and configured to switch off the display screen 65 when the eyesare recognized by the eye detection engine 633 but the face is notrecognized by the face detection engine 631 (e.g., referring to FIG.8A). More specifically, when the eyes are recognized but the face is notrecognized, it means that a user is too close to the display screen 65(or the image system 600) and thus the eye protection engine 635 turnsoff the display screen 65. When the eyes and face are both recognized,it means that the user is at a position farther than the operable rangeDo. Thus the display screen 635 is arranged to display pictures at anormal brightness when (i) the eyes are recognized by the eye detectionengine 633 and (ii) the face is recognized by the face detection engine631 (e.g., referring to FIG. 8B).

In some embodiments, the eye protection engine 635 is integrated in thedisplay screen 65 such that the display screen 65 is aware of thedigital values (e.g., the first control signal S_(c1) and the secondcontrol signal S_(c2)) sent from the face detection engine 631 and theeye detection engine 633 to operate accordingly.

In some scenarios, when a user is too far from the image system 600 orthe ambient light is too dark for the image sensor 61 to capture theface and eyes, the image system 600 identifies that it is not suitablefor a user to watch the pictures shown on the display screen 65, andthus the display screen 65 is closed. Accordingly, the eye protectionengine 635 is further configured to switch off the display screen 65when the face is not recognized or detected by the face detection engine631 and the eyes are also not recognized or detected by the eyedetection engine 633.

In some embodiments, the display screen 65 is not totally turned offwhen a user is too close to the display screen 65. For example, thedisplay screen 65 is configured to display the pictures at a protectionbrightness which is below 30% of the normal brightness when the eyes arerecognized or detected by the eye detection engine 633 but the face isnot recognized or detected by the face detection engine 631. Morespecifically, it is possible that the display screen 65 displayspictures using different brightness according to the recognition resultof the display controller 63 to fulfill different operation scenarios.

As mentioned above, the display controller 63 (including the facedetection engine 631, the eye detection engine 633 and the eyeprotection engine 635) is activated only when a predeterminedApplication is executed or run by the image system 600 so as to reducethe power consumption of the image system 600. The display controller 63may be totally shut down before the predetermined Application isexecuted or run by the image system 600.

As mentioned above, it is possible to determine an operable range Doaccording to the FOV of the image sensor 61. However, since the FOVcannot be changed by the user, it is possible to change the operablerange Do by selecting a WOI from the image Im captured by the imagesensor 61 by a user using a setting stage of a user interface of theimage system 600, wherein the WOI is a pixel region smaller than a sizeof the image Im captured by the image sensor 61 or smaller than a sizeof a sensor array of the image sensor 61. More specifically, the imagesystem 600 has a predetermined operable range Do determined by the FOVof the image sensor 61 before shipment, and a user is able to select anew operable range Do by changing the WOI during operation.

In this embodiment, the face detection engine 631 is configured toselect a window of interest (WOI) in the image Im and recognize whetherthe face is within the window of interest of the image Im. The eyedetection engine 633 is configured to select the window of interest(WOI) in the image Im and recognize whether the eyes are within thewindow of interest of the image Im. The eye protection engine 635 isconfigured to switch off the display screen 65 when the eyes arerecognized within the window of interest of the image Im but the face isnot recognized within the window of interest of the image Im (e.g.,referring to FIG. 8A), and control the display screen 63 to display thepictures at a normal brightness when the eyes are recognized within thewindow of interest of the image Im and the face is also recognizedwithin the window of interest of the image Im (e.g., referring to FIG.8B). The display screen 63 is turned off when both the face and eyes arenot recognized or detected within the window of interest of the imageIm.

Referring to FIG. 9 , it is a flow chart of an operating method of animage system according to one embodiment of the present disclosureincluding the steps of: capturing, by an image sensor, an image havinghundreds of pixels (Step S71); recognizing, by a display controller, aface and eyes in the image (Step S731 and S733); displaying pictures ona display screen when both the face and the eyes are recognized in theimage (Step S751); switching off the display screen by the displaycontroller when the eyes are recognized in the image but the face is notrecognized in the image (Step S753); and switching off the displayscreen by the display controller when both the face and the eyes are notrecognized in the image (Step S755).

Please referring to FIGS. 6, 8A-8B and 9 , details of this embodimentare illustrated hereinafter.

Step S71: The image sensor 61 has a predetermined field of view (FOV)and captures an image Im to be sent to the display controller 63. Asmentioned above, the image sensor 61 of the present disclosure is usedfor controlling the ON/OFF of the image display 65, and it has much lesspixels, e.g., hundreds of pixels, than the camera 69 such that it isturned on to successively capture images Im when the power of the imagesystem 600 is switched on. In some embodiments, the image sensor 61 ofthe present disclosure is not operated before a predeterminedApplication (as mentioned above) is executed by the image system 600,and is activated to start to capture images at a predetermined lowsampling frequency after the predetermined Application is executed.

Steps S731 and S732: The display controller 63 includes a face detectionengine 631 and an eye detection engine 633 configured to respectivelydetect a face and eyes in the image Im captured by the image sensor 61.As mentioned above, the face detection engine 631 and the eye detectionengine 633 are previously trained by machine learning to respectivelyrecognize the face and eyes. The face detection engine 631 outputs afirst control signal S_(c1) (e.g., a digital value of one bit, two ormore bits) indicating whether a face is recognized or detected. The eyedetection engine 633 outputs a second control signal S_(c2) (e.g., adigital value of one bit, two or more bits) indicating whether eyes arerecognized or detected.

The display screen 65 is controlled by the display controller 63. Asmentioned above, in one embodiment the display controller 63 may furtherinclude an eye protection engine 635 to control the ON/OFF of thedisplay screen 65. In another embodiment, the eye protection engine 635is included in the display screen 65 such that the display screen 65receives the first control signal S_(c1) and the second control signalS_(c2) from the display controller 63. The display controller 63 isactivated or turned on only when a predetermined Application (asmentioned above) is executed by the image system 600 to reduce the totalpower consumption. More specifically, the display controller 63 does notperform any calculation even the image sensor 61 captures images Imbefore the predetermined Application is run by the image system 600(e.g., by double clicking by a user). More specifically, it is possiblethat the image sensor 61 and the display controller 63 are not operatedtogether before the predetermined Application is run by the image system600.

Step S751: When the face is detected or recognized by the displaycontroller 63 (e.g., by the face detection engine 631 thereof) in theimage Im and the eyes are detected or recognized by the displaycontroller 63 (e.g., by the eye detection engine 633 thereof) in theimage Im, it means that a user watches the display screen 65 at anappropriate distance and thus the display screen 65 display pictureswith a normal brightness, wherein the normal brightness is set accordingto different designs.

Step S753: When the face (e.g., a complete face shape) is not detectedor recognized by the display controller 63 (e.g., by the face detectionengine 631 thereof) in the image Im but the eyes are detected orrecognized by the display controller 63 (e.g., by the eye detectionengine 633 thereof) in the image Im, it means that a user watches thedisplay screen 65 too close to the display screen 65 and thus displaycontroller 63 turns off the display screen 65 or controls the displayscreen 65 to display pictures with a protection brightness which is lessthan 30% of the normal brightness to protect the user's eyes.

Step S755: When the face (e.g., a complete face shape) is not detectedor recognized by the display controller 63 (e.g., by the face detectionengine 631 thereof) in the image Im and the eyes are also not detectedor recognized by the display controller 63 (e.g., by the eye detectionengine 633 thereof) in the image Im, it means that either ambient lightis too weak or a user does not look at the display screen 65 at all, andthus the display controller 63 turns off the display screen 65. Asmentioned above, the “turning off display screen” herein may beimplemented by several ways as long as the display screen 65 does notsubstantially show any pictures thereon.

Referring to FIG. 10 , it is a flow chart of an operating method of animage system according to an alternative embodiment of the presentdisclosure. The difference between FIG. 9 and FIG. 10 is that in FIG. 10the display controller 63 further selects a window of interest (WOI)from an image Im captured by the image sensor 61 if the FOV of the imagesensor 61 does not fulfill the requirement of the user. That is, in FIG.10 , the operable distance Do is determined by the WOI. Therefore, inFIG. 10 , the display controller 63 is configured to recognize whetherthe face and the eyes are in a window of interest of the image Iminstead of recognizing whether the face and the eyes are in the wholeimage Im (Steps S931 and S933). The display controller 63 is configuredto switch off or reduce the brightness (e.g. with a protectionbrightness) of the display screen 65 when the eyes are recognized in thewindow of interest of the image Im but the face is not recognized in thewindow of interest of the image Im (Step S953); and switch off thedisplay screen 65 when the eyes are not recognized in the window ofinterest of the image Im and the face is also not recognized in thewindow of interest of the image Im (Step S955). The display screen 65 iscontrolled to display pictures at a normal brightness when the eyes arerecognized in the window of interest of the image Im and the face isalso recognized in the window of interest of the image Im (Step S951).In addition to changing the whole image to a window of interest of theimage in the embodiment of FIG. 10 , other details of FIG. 10 aresimilar to those of FIG. 9 and thus details thereof are not repeatedherein.

In another embodiment, the eye protection engine 635 of the displaycontroller does not directly switch off the display screen 65 in theStep 753 of FIG. 9 and the Step S953 in FIG. 10 but to control thedisplay screen 65 to show a notice thereon or to control a speaker ofthe display screen 65 (or display device) to play a voice to warn a userthat he/she is too close to the display screen 65. In this case, thedisplay screen 65 may display with a relatively dark brightness butclearly show the notice with a relatively high brightness. The contentof the notice is a properly designed warning message without particularlimitations, e.g., too close, keep away or the like. The user may clickon the notice or drag the notice out of the display screen 65 to turnoff the warning notice, or the user may press a physical button to turnoff the warning notice. If the notice is not turned off within apredetermined time interval, the eye protection engine 635 of thedisplay controller then switches off the display screen 65.

In some embodiments, if the display controller continuously detects thatthe user is too close to the display screen 65, the notice is shownevery a predetermined time interval, e.g., every 5 minutes, but notlimited thereto.

As mentioned above, the eye protection becomes more and more importantin modern life. Therefore, the present disclosure further provides animage system (FIGS. 6-7 ) and an operating method thereof (FIGS. 9-10 )that switch off or reduce brightness of a display screen when a user istoo close to the display screen according to the appearance of a faceand eyes in a captured image or in a selected window of interest of thecaptured image.

Although the disclosure has been explained in relation to its preferredembodiment, it is not used to limit the disclosure. It is to beunderstood that many other possible modifications and variations can bemade by those skilled in the art without departing from the spirit andscope of the disclosure as hereinafter claimed.

What is claimed is:
 1. An electronic system, comprising: an image sensorhaving a field of view and configured to capture an image; a displayscreen; and a processor electrically coupled to the image sensor and thedisplay screen, and configured to perform a face detection and an eyedetection on the image, perform a notice when the image is recognized tocontain eyes but does not contain a complete face, and switch off thedisplay screen when the notice is not turned off within a predeterminedtime interval.
 2. The electronic system as claimed in claim 1, furthercomprising: a proximity sensor configured to detect whether an object isclose to the electronic system; and a camera configured to capture apicture to be displayed on the display screen or stored in a memory,wherein the image sensor is not included in the proximity sensor or thecamera.
 3. The electronic system as claimed in claim 1, wherein theprocessor is further configured to switch off the display screen when aface and the eyes are not recognized in the image by the processor. 4.The electronic system as claimed in claim 1, wherein the field of viewof the image sensor is previously arranged according to an operablerange of the electronic system.
 5. The electronic system as claimed inclaim 1, wherein the image sensor is turned on full-timely when apredetermined Application is running.
 6. The electronic system asclaimed in claim 1, wherein the processor is further configured toselect a window of interest in the image and recognize whether thecomplete face and the eyes are within the window of interest of theimage, and perform the notice when the eyes are recognized within thewindow of interest of the image but the complete face is not recognizedwithin the window of interest of the image.
 7. The electronic system asclaimed in claim 6, wherein the window of interest of the image ispreviously selected according to an operable range of the electronicsystem.
 8. An electronic system, comprising: an image sensor having afield of view and configured to capture an image; a processorelectrically coupled to the image sensor, and configured to perform aface detection and an eye detection on the image; and a display screenconfigured to display with a brightness when the image is recognized tocontain both eyes and a complete face, and reduce the brightness whenthe image is recognized to contain the eyes but does not contain thecomplete face.
 9. The electronic system as claimed in claim 8, whereinthe display screen is switched off when the eyes and a face are notrecognized in the image by the processor.
 10. The electronic system asclaimed in claim 8, wherein the field of view of the image sensor ispreviously arranged according to an operable range of the electronicsystem.
 11. The electronic system as claimed in claim 8, furthercomprising: a proximity sensor configured to detect whether an object isclose to the electronic system; and a camera configured to capture apicture to be displayed on the display screen or stored in a memory,wherein the image sensor is not included in the proximity sensor or thecamera.
 12. The electronic system as claimed in claim 8, wherein theprocessor is further configured to select a window of interest in theimage and recognize whether the complete face and the eyes are withinthe window of interest of the image, and the display screen isconfigured to display with the brightness when (i) the eyes arerecognized within the window of interest of the image and (ii) thecomplete face is recognized within the window of interest of the image.13. The electronic system as claimed in claim 12, wherein the window ofinterest of the image is previously selected according to an operablerange of the electronic system.
 14. An electronic system, comprising: animage sensor having a field of view and configured to capture an image;a display screen; and a processor electrically coupled to the imagesensor and the display screen, and configured to perform a facedetection and an eye detection on the image, and control a speaker toplay a warning voice when the image is recognized to contain eyes butdoes not contain a complete face.
 15. The electronic system as claimedin claim 14, further comprising: a proximity sensor configured to detectwhether an object is close to the electronic system; and a cameraconfigured to capture a picture to be displayed on the display screen orstored in a memory, wherein the image sensor is not included in theproximity sensor or the camera.
 16. The electronic system as claimed inclaim 14, wherein the processor is further configured to switch off thedisplay screen when the complete face and the eyes are not recognized inthe image.
 17. The electronic system as claimed in claim 14, wherein theimage sensor is turned on full-timely when a predetermined Applicationis running.
 18. The electronic system as claimed in claim 14, whereinthe processor is further configured to select a window of interest inthe image and recognize whether the complete face and the eyes arewithin the window of interest of the image, and control the speaker toplay the warning voice when the eyes are recognized within the window ofinterest of the image but the complete face is not recognized within thewindow of interest of the image.
 19. The electronic system as claimed inclaim 18, wherein the window of interest of the image is previouslyselected according to an operable range of the electronic system. 20.The electronic system as claimed in claim 14, wherein the field of viewof the image sensor is previously arranged according to an operablerange of the electronic system.